The present invention relates to optical associative parallel computing configurations. More specifically the present invention relates to implementation of general associative processing modules which adapt in real time as they are exposed to associated information patterns, (u,v), so at a later time presentation of one pattern, u, results in recall of the other, v.
There has been a recent resurgence of interest in associative-network parallel processing architectures, see T. Kohonen, Self-Organization and Associative Memory (Springer Verlag, New York, 1984); G. E. Hinton and J. A. Anderson, Parallel Models of Associative Memory (Laurence Erlbaum Associates, Hillsdale, NJ, 1981); A. C. Sanderson and Y. Y. Zeevi, Eds., "Special Issue on Neural and Sensory Information Processing", IEEE Trans, Syst. Man and Cybernetics, SMC-13 (1983); J. J. Hopfield, Proc. Natl. Acad., Sci., 79, p. 2554 (1982) and 81, p. 3088 (1984); and S. Grossberg, Studies of Mind and Brain (R. Reidel Publ. Co., Boston, 1982); for applications where the acquisition, transformation, matching/recognition, and manipulation of large amounts of information in parallel are required; for example, in pattern recognition, image understanding, robotics, expert systems, and a variety of other artificial intelligence problems. Associated information patterns u, v that are manipulated by the parallel processing architectures could encode, for example: pixel intensities from a 2-D image, strings of characters from a textual database, logical inferences, feature vectors, speech spectral samples, the state variables of a control system, antenna array signals, or the output of robot sensors. An individual associative module, or building block, which adapts in real time can potentially perform a variety of data manipulations and computations in addition to its basic associative memory function. For example, the output could be a vector v representing a spatial transformation of, or an array of features extracted from, an input image u. Nonlinearity, such as thresholding of the output v, allows yes/no decisions to be made and can force a learned pattern to be exactly recalled, rather than a superposition of learned patterns, when a distorted pattern is presented. By exploiting the ability to recall information with a partial or distorted key, u, it may be possible to implement invariances, for example to rotation, translation, or scaling of objects in an input scene. Similarly, patterns such as handwritten characters could be reduced to a standard form or symbolic code. By reconstructing an output pattern v from a partial description or related input pattern u, the associative element can "recognize" patterns with similar meaning. An associative module can also function as a novelty filter, which responds only to patterns that have not been previously encountered. By feeding portions of the output pattern back to the input, time sequences of patterns v(t) can be recalled or "semantic nets" of interrelated patterns can be reconstructed over time. Significantly greater computational abilities can be obtained by interconnecting and cascading associative modules.
There are a large number of potential implementations for an associative module, including: content-addressable digital memory, linear algebra matrix approaches, holography, and neural networks. The various associative schemes offer differing performance capabilities, but they all generally require massively parallel information processing in order to function effectively. Obtaining the required degree of parallelism has generally be a formidable block to the development and practical application of associative processing architectures. For example, a neural network capable of storing m(.ltoreq.n) pairs of n element vectors requires n processing elements ("neurons") operating in parallel, with each element performing n multiplications and n additions in parallel, for a total of n.sup.2 parallel multiplications and additions. A linear algebraic associative module, performing recall with the matrix-vector product of Eq. (1) below, has the same amount of intrinsic parallelism, and an efficient implementation must perform n.sup.2 parallel multiplications and additions. An efficient content-addressable digital memory should simultaneously compare the current input u against all m possible matching u patterns. Conventional analog or digital approaches, employing wires, resistors and/or integrated circuits, cannot generally fully implement the required parallelism.
Optical associative modules, generally holographic, see D. Gabor, Nature, 206, p. 422 (1965) and IBM J. Res. Develop,. p. 156 (1969); A. W. Lohmann and H. W. Werlich, Phys, Lett., 25A, p. 570 (1967), and A. W. Lohmann and Ch. Thum, Opt. Com., 46, p. 74 (1983); D. J. Willshaw, O. P. Buneman and H. C. Longuet-Higgins, Nature, 222, p. 960 (1969), and D. J. Willshaw and H. C. Longuet-Higgins, Machine Intelligence, 5, p. 351, (1970); M. Nakajima, T. Morikawa, and K. Sakurai, Appl. Opt., 11, p. 362 (1972), and H. Akahori and K. Sakurai, Appl. Opt. 11, p. 413 (1972); H. Mada, Appl. Opt., 24, p. 2063 (1985); or linear-algebriac, see D. Psaltis and N. Farhat, Digest of the 13th Congress of the ICO, ICO-13, Sapporo, Japan, p. 24 (1984); D. J. Condon, M. C. Reichenbach, A. Tarasevich, and W. T. Rhodes, Postdeadline Paper, OSA Topical Meeting on Optical Computing (1985); G. Eichmann and H. J. Caulfield, Appl. Opt., 24, p. 2051 (1985): have been proposed. However, those have lacked the ability to adapt and thereby learn new associations in real time; many have been read-only memories, where associations were computed offline or created in a separated process in the form of a fixed optical mask or transparancy. Adaptivity is essential to fully exploit the capabilities of associative processing. The information required for successful performance in many applications, such as image understanding, is often too extensive and detailed to be pre-specified; adaptive learning provides a more practical means for selecting and acquiring this knowledge.